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ABSTRACT 

An  Expert  system  for  Shipboard 
Obscuration  Prediction  (AESOP),  an 
Artificial  Intelligence  (AI)  approach  to 
forecasting  maritime  visibility  obscura¬ 
tions,  has  been  designed,  developed,  and 
tested.  AESOP  is  rule-based,  using 
backward  chaining.  The  current  version, 
AESOP  2.1,  has  290  rules  and  has  been 
designed  in  terms  of  nowcasts  (0-1  hr)  and 
forecasts  (1-6  hr).  An  extensive  explana¬ 
tion  feature  allows  the  user  to  understand 
the  reasoning  process  behind  a  particular 
forecast.  AESOP  has  been  evaluated 
against  100  independent  test  cases,  in 
which  clear,  hazy,  or  foggy  conditions  are 
predicted.  The  overall  performance  of 
AESOP  is  68%  correct.  This  value 
indicates  considerable  forecast  skill  when 
compared  to  36%  for  random  chance.  When 
the  distinction  between  clear  and  haze  is 
ignored,  the  expert  system  correctly 
forecasts  79%  of  the  "Fog"/"No  fog" 
situations . 


1 .  Introduction 

For  ships  at  sea,  a  visibility  re¬ 
striction  poses  just  as  serious  an 
obstruction  to  movement  as  it  does  over 
land  or  in  the  air.  In  military  opera¬ 
tions,  ship  movement  often  occurs  simulta¬ 
neously  with  aircraft  flights,  as  is  the 
case  with  aircraft  carriers.  For  this 
reason,  and  because  ship  operations  can  be 
moved  from  one  location  to  another  to  take 
advantage  of  more  favorable  weather  condi¬ 
tions,  the  accurate  prediction  of  maritime 
visibility  has  been,  and  continues  to  be, 
an  important  problem  for  the  Navy. 

The  Navy  is  developing  a  shipboard, 
environmental  diagnosis/forecast  system 
called  the  Tactical  Environmental  Support 
System  (TESS).  Among  the  several  purposes 
of  TESS  is  the  concept  of  bringing  automa¬ 
tion  and  advanced  analysis  capabilities  to 
the  Navy  shipboard  oceanographer/meteorol¬ 
ogist,  primarily  on  aircraft  carriers  and 
other  large  ships. 

NOARL  contribution  no.  90:073:401. 
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Expert  systems  (ES)  have  been  pro¬ 
posed  for  TESS  problems  requiring  exper¬ 
tise  in  a  well-defined  domain,  including 
the  forecasting  of  visibility  obscurations 
(e.g.  fog  and  haze.)  The  specific  goal  is 
to  develop  a  system  that  provides  a  mari¬ 
time  obscuration  forecast  that  follows  a 
reasoning  process  similar  to  that  of 
expert,  human  forecasters. 

The  purpose  of  this  paper  is  to 
summarize  the  rationale  and  method  that 
has  gone  into  the  2  1/2-year  development 
of  the  Navy  ES  for  forecasting  obscura¬ 
tions  at  sea.  The  reader  is  referred  to 
Peak  and  Tag  [1]  for  a  more  complete 
description  of  the  system  design  and 
development.  In  this  paper  a  new,  inde¬ 
pendent  test  of  the  system's  performance 
not  yet  completed  at  the  time  of  publica¬ 
tion  of  Peak  and  Tag  [1]  will  be 
presented . 

2.  Expert  system  approach 

The  field  of  Artificial  Intelligence 
(AI)  includes  a  number  of  techniques  for 
solving  problems  that  involve  reasoning 
about  data  and  reaching  conclusions. 

Expert  Systems  have  emerged  as  one  of  the 
applications  of  AI  technology  to  real- 
woVld  problems.  ESs  are  AI  computer 
programs  that  perform  inference  processes 
based  on  a  collection  of  expertise  and  a 
set  of  known  facts  about  the  situation  at 
hand.  This  procedure  may  be  based  on  both 
formal  knowledge  and  heuristics,  and  the 
problem-solving  procedure  may  differ  for 
various  sets  of  input  data. 

The  TESS  requirement  is  that  the 
obscuration  expert  system  predict  fog  and 
haze  for  any  maritime  location  between 
70°N  and  70°S.  Visibility  obscuration  due 
to  precipitation  is  not  a  designated 
function  of  the  system.  The  ES  is  intend¬ 
ed  to  be  run  onboard  Navy  ships  using  data 
available  from  local  measurements,  TESS 
data  fields  and  satellite  images. 

There  are  several  advantages  to  the 
use  of  an  ES  for  the  prediction  of  such 
obscuration  phenomena.  The  prediction  of 
obscuration  events  such  as  fog  and  haze  is 
a  very  difficult  problem  requiring  the 
interpretation  of  many  types  of  data. 

Since  fog  may  form  by  several  processes 
[2J,  there  is  no  pre-defined  algorithm  for 
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its  prediction.  Thus,  the  ES  approach  is 
useful  because  an  ES  can  be  made  to  in¬ 
clude  expertise  on  how  to  approach  the 
forecast  problem  based  on  the  situation  at 
hand.  The  expertise  to  forecast  maritime 
fog  is  a  rare  commodity.  Since  Navy 
forecasters  tend  to  serve  relatively  short 
duty  tours  at  sea,  they  may  not  have  the 
time  to  develop  an  expert  level  ..f  skill. 
By  encoding  available  forecast  knowledge 
into  an  ES,  rare  expertise  can  be  dissemi¬ 
nated  to  the  fleet. 

The  ES  developed  in  this  project  has 
been  named  AESOP  (An  Expert-system  for 
Shipboard  Obscuration  Prediction).  The 
main  source  of  expertise  for  this  study 
was  a  series  of  research  reports  (e.g. 
(3,4))  from  the  Calspan  Corporation  de¬ 
tailing  the  results  of  maritime  fog  stud¬ 
ies  during  1972-1983.  In  addition,  one  of 
the  participants  in  these  studies,  C.  W. 
Rogers,  has  acted  as  a  consultant  expert 
to  provide  input  in  the  development  of  the 
AESOP  rule  base. 

3.  The  AESOP  system 

In  this  section  the  design  of  AESOP 
will  be  summarized.  A  complete  descrip- 
l ion  of  AESOP  is  presented  in  Peak  and  Tag 

I  1  1  • 

A  human  expert  uses  a  complicated 
reasoning  process  to  forecast  maritime 
obscurations.  Typically  one  must  follow 
multiple  lines  of  reasoning  using  many 
different  types  of  data.  AESOP  has  been 
designed  to  evaluate  forecast  situations 
in  a  fashion  similar  to  that  of  expert 
human  forecasters. 

The  problem-solving  paradigm  for 
AESOP  is  a  consultation  session  in  which 
the  user  of  the  program,  a  Navy  meteorolo¬ 
gist,  answers  questions  about  various 
atmospheric  parameters  such  as  tempera¬ 
ture,  dewpoint,  sea  surface  temperature, 
etc.  Once  AESOP  has  acquired  enough 
information  about  the  current  condition  of 
the  atmosphere,  it  applies  the  expertise 
contained  in  its  knowledge  base  to  draw 
conclusions  about  the  future  condition  of 
the  atmosphere  with  regard  to  visibility 
obscuration . 

The  knowledge  and  expertise  in  AESOP 
are  stored  as  a  series  of  IF-THEN  rules. 
These  rules  contain  the  basic  knowledge 
concerning  the  relationships  of  facts 
aboat  the  problem  domain.  These  rules  are 
the  major  component  of  the  ES,  and  they 
form  the  basis  for  the  formal  reasoning 
process  that  the  system  uses  to  solve 
problems . 

AESOP  is  implemented  in  the  Prolog 
language  on  an  IBM-compatible  personal 
computer.  The  current  version,  AESOP  2.1, 
has  290  rules.  AESOP  has  been  di  -igned  to 
reason  in  terms  of  nowcasts  and  l..iecasts. 
The  AESOP  nowcasts  are  for  the  0- i  h  time 
frame,  ’-’bile  the  AESOP  forecasts  apply  to 
the  1-6  h  time  frame.  This  approach  is 


used  to  differentiate  between  situations 
where  a  change  in  the  atmospheric  obscura¬ 
tion  condition  is  imminent,  and  those 
where  the  condition  change  requires  more 
time  to  occur. 

Since  one  of  the  goals  of  AESOP  is  to 
disseminate  rare  expertise,  the  predic¬ 
tions  are  accompanied  by  a  synopsis  of  the 
physical  reasoning  used  in  arriving  at  the 
prediction.  AESOP  also  includes  an  exten¬ 
sive  explanation  feature.  The  user  is 
able  to  step  through  the  reasoning  proc¬ 
ess,  during  which  AESOP  reveals  the  logic 
by  which  its  conclusions  were  made  and 
also  the  reasons  that  alternative  conclu¬ 
sions  were  not  made.  Thus,  it  effectively 
tells  the  user  why  a  certain  obscuration 
is  expected,  why  other  obscurations  are 
not  expected,  and  what  data  were  used  be 
make  these  conclusions. 

The  major  components  of  the  AESOP 
expert  system  (Fig.  1)  include  the  Knowl¬ 
edge  Base  (containing  both  the  Working 
Memory  and  the  Rule  Base),  the  Fact  Acqui¬ 
sition  System,  the  Explanatory  Interface, 
the  Inference  Engine  and  the  User  Inter¬ 
face.  The  User  Interface  makes  communica¬ 
tion  between  the  user  and  AESOP  possible. 
The  Fact  Acquisition  System  systematically 
makes  inquiries  to  the  user  concerning 
atmospheric  parameters  (e.g.  temperature, 
wind  speed,  etc.)  and  records  this  infor¬ 
mation  as  facts  in  the  Working  Memory. 

The  Inference  Engine  applies  rules  of 
logic  to  infer  new  facts  from  the  existing 
facts.  The  Rule  Base  contains  the  static 
knowledge  previously  obtained  from  the 
expert  sources  in  the  form  of  rules  of 
logic.  The  Working  Memory  contains  the 
facts  that  describe  what  is  known  about  a 
particular  problem.  When  the  program 
starts,  the  Working  Memory  is  empty.  The 
dynamic  knowledge  obtained  from  the  user 
via  the  Fact  Acquisition  System  is  stored 
in  the  Working  Memory.  As  intermediate 
conclusions  are  made  via  the  Inference 
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Figure  1.  Major  components  of  the  AESOP 
expert  system.  Arrows  indicate  data  flow. 
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Engine,  the  system  stores  this  new  knowl¬ 
edge  in  the  Working  Memory.  Finally,  the 
Explanatory  Interface  allows  the  user  to 
step  through  the  many  logical  paths  used 
by  AESOP  to  arrive  at  its  forecasts. 

The  Prolog  language  is  designed  to 
use  backward  chaining  [5]  to  try  to  verify 
previously  specified  goals.  In  AESOP, 
these  goals  are  specified  based  on  the 
current  state  of  the  atmosptiere  with 
regard  to  visibility-obscuring  phenomena. 
There  are  three  mutually-exclusive  atmos¬ 
pheric  states  that  AESOP  diagnoses:  Fog  is 
present;  Haze  is  present;  and  No  Obscura¬ 
tion  is  pinsent.  The  initial  state  of  the 
atmosphere  is  one  of  these  three,  and  the 
forecast  future  state  will  also  be  one  of 
these  three  possibilities  (Table  1).  The 
connections  between  th»  future 

states  are  various  meteorological  process¬ 
es.  These  connections  have  been  designat¬ 
ed  as  the  following  state-change  opera¬ 
tors:  1)  Fog  will  form,  2)  Fog  will 

dissipate,  3)  Fog  will  persist,  4)  Haze 
will  form,  5)  Haze  will  dissipate,  6)  Haze 
will  persist,  and  7)  It  will  stay  clear. 
AESOP  is  designed  to  analyze  the  likeli¬ 
hood  of  these  state  changes  based  on 
observed  meteorological  parameters. 

For  any  given  initial  state,  only 
three  of  these  state-change  operators  may 
apply  (Table  1).  The  AESOP  Inference 
Engine  uses  the  current  obscuration  and  a 
set  of  lists  similar  to  Table  1  to  deter¬ 
mine  which  three  state-changes  to  test. 
These  three  state  changes  are  then  desig¬ 
nated  as  the  goals  of  the  ES .  This  ini¬ 
tial  limitation  of  the  number  of  goals 
accomplishes  a  heuristic  reduction  of  the 
search  space. 


Table  1.  Current  and  future  atmospheric 
obscuration  states.  The  three  possible 
future  states  and  the  corresponding  state- 
change  operators  are  indicated  for  each 
initial  state. 

Current  State:  No  Obscuration 

Future  States _ State-Change  Operators 

No  Obscuration  It  will  stay  clear 

Haze  Haze  will  form 

Fog  Fog  will  form 

Current  State:  Haze 

Future  States _ State-Change  Operators 

Ho  Obscuration  Haze  will  dissipate 

Haze  Haze  will  persist 

Fog  log  will  form 

Current  State:  Fog 

Future  States _ State-Change  Operators 

Ho  Obscuration  Fog  will  dissipate 

Haze  Haze  will  form 

Fog  Fog  will  persist 


As  AESOP  evaluates  each  potential 
goal  state,  a  probability  of  its  occui - 
rence  is  assigned.  In  general,  the  goal 
state  with  the  highest  probability  is 
chosen  to  be  the  AESOP  forecast.  However, 
since  fog  is  the  most  severe  type  of 
visibility  obscuration,  whenever  the 
probability  of  its  occurrence  is  greater 
than  50%,  AESOP  selects  fog  as  its  fore¬ 
cast  even  if  the  probability  of  haze  is 
larger. 

"Semantic  nets"  (5]  represent  a 
knowledge  domain  by  a  graphic  collection 
of  nodes  and  links  where  the  nodes  repre¬ 
sent  objects  or  concepts  and  the  links 
represent  relationships  between  the  ob¬ 
jects  or  concepts.  Knowledge  in  AESOP  is 
represented  by  rules.  A  set  of  rules  may 
represented  in  an  inference  net, 
which  is  a  special  type  of  semantic  net¬ 
work  using  only  the  IF-THEN  logic  rela¬ 
tion.  Thus,  portions  of  the  AESOP  rule 
base  may  be  presented  in  graphic  form. 

The  inference  net  depicting  the  rules 
that  determine  the  likelihood  of  forecast 
(1-6  hr)  fog  formation  is  presented  in 
Figure  2.  The  nodes  (boxes)  in  Figure  2 
represent  goals  and  subgoals.  Dashed 
links  are  defined  here  as  “OR"  links 
because  the  goal  to  which  they  lead  suc¬ 
ceeds  when  any  one  of  the  OR-links  pro¬ 
ceeds  from  a  true  subgoal.  All  of  the 
solid  AND-links  must  proceed  from  true 
subgoals  before  the  subgoal  to  which  they 
lead  succeeds.  This  inference  net  depicts 
the  rule 

IF  Fog  forms  by  the  Taylor  process 

OR  Fog  forms  by  the  stratus-lowering 

process 

OR  There  is  advection  of  existing  fog 
THEN  "Fog  will  form"  is  forecast. 

Similarly,  the  rule 

IF  The  marine  layer  is  primed  for  fog 

AND  The  marine  layer  is  cooled  from 

below  to  dewpoint 

AND  The  predominant  flow  is  from  warmer 
to  colder  water 

THEN  Fog  forms  by  the  Taylor  process 

is  also  depicted.  Thus,  the  inference  net 
in  Figure  2  reveals  the  fog  formation 
rules  and  their  interdependence. 

The  major  subgoals  in  Figure  2  are 
the  different  formation  processes  for 
Taylor  and  stratus-lowering  fog,  and  for 
advection  of  existing  fog.  AESOP  attempts 
to  verify  the  goal  "Fog  will  form"  by 
chaining  backward  througli  ttie  subgoals 
"Fog  forms  by  the  Taylor  process,"  "Fog 
forms  by  the  stratus-lowering  process"  and 
"There  is  advection  of  existing  fog." 

Each  of  these  subgoals  must  itself  be 
verified  by  determining  the  truth  of  the 
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Figure  2.  Inference  network  depicting  the 
AESOP  rules  for  the  forecast  goal  "Fog 
will  form."  Dashed  lines  are  OR-links  and 
solid  lines  are  AND-links.  Nodes  which 
have  no  link  leading  to  them  are  verified 
based  on  user-input  data  in  the  Working 
Memory.  Backward  chaining  progresses  from 
top  to  bottom  whereas  inferences  are  made 
from  the  bottom  to  the  top. 


subgoals  upon  which  it  depends.  The 
search  for  a  solution  proceeds  down  the 
net  until  there  are  no  further  branches. 

At  this  point,  the  last  subgoal  must  be 
verified  by  the  appropriate  data  values 
(not  shown)  previously  input  by  the  user 
via  the  Fact  Acquisition  System. 

Similar  inference  nets  (not  shown) 
may  be  constructed  from  the  AESOP  rules 
for  fog  dissipation,  and  those  for  haze 
formation  and  dissipation.  A  complete  set 
of  the  AESOP  2.1  inference  nets  is  pre¬ 
sented  in  Peak  (6). 


The  explanatory  interface  in  many  ESs 
is  little  more  than  a  procedure  to  list 
rules  that  were  used  to  reach  a  goal.  The 
AESOP  explanation  feature  is  expanded  to 
include  explanatory  text  that  reveals  the 
motivations  behind  the  different  lines  of 
reasoning,  the  physical  causes  and  effects 
underlying  the  obscuration  forecasts,  the 
data  values  used  and  why  the  data  are 
important.  Another  major  difference  in 
AESOP's  explanation  feature  is  that  it 
reveals  not  only  the  lines  of  reasoning 
that  succeed  but  also  those  that  fail. 

The  advantage  in  this  approach  is  that  a 
user  may  be  just  as  interested  in  why 
AESOP  did  not  forecast  an  obscuration  that 
he  may  have  thought  to  be  likely,  as  he  is 
interested  in  why  AESOP  did  forecast  an 
event  he  thought  would  not  happen. 

AESOP  does  this  complete  evaluation 
by  i esting  every  possible  line  of  reason¬ 
ing  to  the  fullest,  regardless  of  the 
success  or  failure  to  meet  the  logical 
requirements  of  each  rule.  In  most  ESs, 
a  particular  chain  of  reasoning  (e.g.  a 
path  from  the  data  to  a  goal  on  an  infer¬ 
ence  net)  is  tested  only  to  the  point 
where  one  of  the  subgoals  fails.  At  this 
point,  the  system  backtracks  and  tries  a 
different  path  until  eventually  a  complete 
path  is  found.  AESOP,  however,  was  de¬ 
signed  to  record  its  paths  continually  in 
the  Working  Memory,  keeping  track  of  the 
success  or  failure  of  the  data  to  meet  the 
requirements  of  each  subgoal.  Thus,  the 
AESOP  Inference  Engine  attempts  to  satisfy 
all  of  the  paths  between  the  data  and  the 
goals.  Even  when  a  subgoal  fails,  the 
remainder  of  the  path  is  still  tested. 

The  difference  is  that  the  path  record  in 
the  Working  Memory  is  flagged  as  not 
satisfying  the  subgoal.  The  explanatory 
text  that  is  generated  by  successful 
subgoals  is  different  from  that  generated 
when  a  subgoal  fails.  Thus,  a  complete 
explanation  of  all  lines  of  reasoning  is 
available. 

One  disadvantage  of  this  approach  is 
that  it  may  take  a  long  time  to  traverse 
all  of  the  branches  of  a  large  rule  base. 
However,  the  AESOP  rule  base  is  not  exces¬ 
sively  large,  plus  the  search  space  lias 
been  further  reduced  by  the  use  of  meta¬ 
rules  . 

The  Explanatory  Interface  is  a  rou¬ 
tine  that  steps  through  these  path  records 
in  the  Working  Memory.  The  path  records 
are  stored  in  tree  form  with  nodes  con¬ 
taining  a  description  of  the  subgoal ,  the 
explanatory  text,  and  pointers  to  the 
node's  parent  and  successor  nodes.  The 
Explanatory  Interface  displays  this  infor¬ 
mation  and  allows  the  user  to  traverse  the 
tree  to  discover  the  cause  and  effect  of 
each  subgoal. 
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The  current  form  of  explanation 
provides  the  most  detailed  window  on  the 
reasoning  process  available  in  ES  technol¬ 
ogy.  AESOP  also  includes  an  explanation 
summary  to  reveal  a  simple  overview  of  the 
reasoning  behind  the  forecast  choice. 

This  less  detailed  general  explai  it.ion 
accompanies  the  initial  AESOP  fo,.:cast. 

The  user  can  still  optionally  choose  to 
delve  into  ttie  detailed  explanation  when 
the  more  complete  line  of  reasoning  is 
desired. 

4.  System  performance 


sor  lowrrosr 


>  rar  mill  >09* 

Tliorp  }•  4  feoltJ  siraiut  4ccfc.  flu  Inversion  Is  turn  S 
Is  sliallou  enousli  for  fog.,  tliusi  ths  vertical  tonyerat 
cnnduclvo  to  stratus- lovor Ins  fo9»  Insolation  on  llm  s 
tliusi  stratus- lover  Ins  fos  Is  Directed  to  devolopi 

fliise  are  the  forecasts  for  etch  potential  obscuration* 

!‘og.  tyobi  95H  ICMnatolojIcal  fos  rrobabU|Ut 

rrob»  IBB*  IfstFnated  vlilbllUv*  6  nllcsl 
lo  Obscuratlom  frob'  6 a 


turn  9.  Tl>s  narlne  feyer 
nrcraturo  structure  Is 
tlio  stratus  Is  snail. 


- — - 4*  hi  Id  >700  like  ne  to 

(li|>l«i|  the  imvcitt 

To)  low  ll>0  (orecul  chain  nr  reamnlnf 

rmaifij  firtnelrrt  for  thfa  case  I  rrrur 

Sun  a  imtu  case 

F»ll  r run  flfSOf  forecast 


A  complete  AESOP  forecast  takes  only 
about  2  min  of  real  time  to  execute.  Most 
of  that  1 i me  is  spent  responding  to  the 
queries  of  the  Fact  Acquisition  System. 

An  example  of  an  AESOP  forecast  display  is 
presented  in  Figure  3.  In  this  case,  fog 
is  forecast  to  occur  with  a  probability  of 
90%.  From  the  summary  explanation  of  the 
reasoning  process,  the  user  can  determine 
that  the  fog  is  expected  to  form  via  the 
stratum-lowering  process.  Ttie  forecasts 
for  all  three  potential  obscurations  are 
listed  next  for  comparison.  Finally,  an 
options  list  gives  he  user  several 
choices  of  what  to  do  next.  First,  he  may 
want  to  see  the  0-1  h  nowcast.  The  second 
option  is  to  traverse  the  complete  expla¬ 
nation  tree  so  that  the  reasoning  behind 
any  of  the  expected  or  not  expected  proc¬ 
esses  is  revealed.  AESOP  also  includes  a 
feature  by  which  one  or  more  of  the  data 
values  previously  input  by  ttie  user  can  be 
modified  and  a  new  forecast  generated. 

This  feature  enhances  the  role  of  the 
system  as  a  training  tool  because  the  user 
can  compare  what  happens  under  slightly 
different  conditions.  Finally,  the  user 
can  run  a  completely  new  case  or  exit  from 
the  forecast  mode  altogether. 

In  this  section,  the  AESOP  forecast 
skill  is  evaluated  for  100  maritime  obscu¬ 
ration  situations.  The  test  data  are 
taken  from  various  weather  ships  stationed 
in  the  North  Pacific  and  North  Atlantic 
during  1971-1974.  Because  fog  is  an 
infrequent  phenomenon,  the  selection  of 
cases  for  the  independent  sample  was  not 
made  randomly.  Instead,  available  ship 
data  were  scanned  to  find  situations  where 
fog  and  haze  were  forming,  persisting,  or 
dissipating.  The  cases  were  not  selected 
on  the  basis  of  their  being  unusual  nor 
overly  simple;  the  only  consideration  was 
whether  fog  or  haze  was  involved  either 
initially  or  at  +6  h.  No  cases  were 
included  that  were  clear  at  both  the 
initial  time  and  at  +6  h. 

Contingency  tables  (e.g..  Table  2) 
are  used  to  compare  the  +6  h  forecasts  to 
the  actual  obscuration  state.  Tim  columns 
represent  the  actual  +6  h  obscuration 
states  while  the  rows  are  the  AESOP  +6  h 
forecasts.  If  the  cases  were  all  Correct- 


Figure  3.  Example  AESOP  forecast  display. 


ly  forecast,  only  the  diagonal  values 
(bold  numbers )  would  be  iionC'ir.  since 
the  extent  of  obscuration  severity  ranges 
from  fog  to  haze  to  clear  in  Table  2,  the 
incorrect  forecasts  above  the  diagonal 
indicate  over  forecasts ,  because  a  more 
severe  visibility  obstruction  is  forecast 
than  actually  occurs.  Similarly,  those 
cases  below  the  diagonal  are  underfore¬ 
casts  . 

The  results  of  the  AESOP  2.1  runs  on 
the  100  independent  sample  cases  are 
presented  in  Table  2.  The  initially  fog 
cases  (Table  2a)  are  correctly  forecast 
59%  of  the  time.  If  persistence  were 
used,  only  16  forecasts  (35%)  would  be 
correct  so  that  the  AESOP  forecasts  do 
seem  to  indicate  skill. 

The  contingency  table  for  the  31 
initially  haze  cases  is  presented  in  Table 
2b.  AESOP  correctly  forecasts  68%  cor¬ 
rect.  If  persistence  were  used,  only  26% 
would  be  correct.  Notice  that  on  these 
cases,  there  is  no  overforecast  or  under¬ 
forecast  bias  (Table  2b). 

The  AESOP  performance  on  the  initial¬ 
ly  clear  cases  (Table  2c)  is  excellent, 
with  87%  correct.  One  should  not  use 
persistence  as  a  comparison  here  because 
this  set  was  deliberately  chosen  not  to 
include  cases  that  were  clear  at  +6  h. 

When  all  100  cases  are  considered 
together  (Table  2d),  AESOP  forecasts  68% 
correct.  For  the  same  reason,  persistence 
can  not  be  used  as  a  comparison  method 
here.  There  is  no  strong  tendency  to 
overforecast  (14  cases)  or  to  underfore¬ 
cast  (18  cases)  the  independent  sample 
cases.  The  random  simulations  described 
in  the  next  section  indicate  that  a  purely 
random  forecast  would  result  in  only  36% 
correct  for  these  cases. 

It  is  useful  to  consider  the  forecast 
skill  in  a  "Fog"/"No  fog"  sense  so  that 
the  more  important  fog  forecast  situations 
are  emphasized  by  excluding  the  less 
Important  haze  situations.  AESOP  correct¬ 
ly  forecasts  79%  of  these  cases.  The 
nearly  80%  skill  of  these  AESOP  2.1  fore¬ 
casts  is  highly  encouraging.  The  random 
simulations  described  in  the  next  section 
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Table  2.  Contingency  tables  for  AESOP  2.1 
+6  h  forecasts  (Feat)  (rows)  vs.  +6  h 
actual  obscuration  states  (columns)  for 
cases  with  the  initial  condition:  a.  fog 
(F),  b.  haze  (H),  c.  clear  (C),  d.  all 
cases,  e.  fog/no  fog.  Bold  values  on 
diagonal  indicate  numbet  of  correct  fore¬ 
casts.  Tot  indicates  total  of  rows  or 
columns,  %  indicates  percent  correct. 
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indicate  that  a  purely  random  forecast 
would  result  in  approximately  50%  correct 
for  these  cases. 


5.  Monte  Carlo  Significance  Tests 

In  an  attempt  to  measure  the  signifi¬ 
cance  of  these  statistics,  Monte  Carlo 
simulations  have  been  used  (e.g..  Peak  and 
Tag,  1989).  The  method  is  to  use  the 
probability  of  occurrence  of  the  three 
obscuration  states  in  Table  2d,  and  the 
probability  of  AESOP  forecasts  for  the 
three  states  from  the  same  table.  A 
random  number  generator  based  on  these 
probabilities  selects  a  random  obscuration 
state  and  a  random  forecast  of  that  state. 
The  process  is  repeated  100  times  to 
simulate  the  AESOP  forecasts  for  a  Lest 
sample  the  same  size  as  the  independent 
sample.  The  reason  this  simulation  can 
measure  the  significance  level  is  that  it 
is  known  that  the  percent  correct  is 
achieved  purely  by  chance.  If  a  random 
percent  correct  could  have  occurred  by 
chance,  it  is  less  likely  that  the  demon¬ 
strated  AESOP  performance  is  due  to  skill. 

The  random  experiment  was  repeated 
50,000  times,  which  is  arbitrarily  chosen 
to  be  enough  trials  to  generate  a  distri¬ 
bution  of  the  random  forecast  skill.  Of 
the  50,000  trials,  the  68  correct  achieved 
in  the  Table  2d  results  is  never  accom¬ 
plished.  The  highest  random  skill  level 
is  57  correct.  Thus,  it  is  virtually 
assured,  at  least  in  a  statistical  sense, 
that  the  AESOP  results  are  due  to  forecast 
skill . 


6.  Summary  and  conclusions 


An  Expert  system  for  Shipboard 
Obscuration  Prediction  (AESOP)  has  been 
developed  over  the  past  2  1/2  years  to 
provide  an  Artificial  Intelligence 
approach  to  short-term  forecasting  of  fog 
and  haze.  In  this  report,  the  latest 
version  (2.1)  is  described  and  an  evalua¬ 
tion  of  the  system  on  an  independent  data 
set  is  made. 

In  a  100-case  independent  sample 
test,  AESOP  is  correct  68%  of  the  time. 
This  value  compares  favorably  with  the  36% 
correct  due  to  random  forecasts  based  on 
the  same  sample  characteristics. 

When  the  distinction  between  haze  and 
clear  is  removed,  AESOP  correctly 
forecasts  79%  of  the  "Fog"/”No  fog" 
situations.  This  level  is  considerably 
higher  than  the  approximately  50%  correct 
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The  independent  sample  results  show  a 
slight  decrease  in  performance  from  the 
dependent  sample  results  in  [1].  This 
decrease  may  be  an  indication  that  the 
rule  base  has  been  "tailored"  to  handle 
cases  with  the  characteristics  of  the 
dependent  sample  because  any  deficiencies 
in  the  rule  base  concerning  those  situa¬ 
tions  have  shown  up  during  earlier  depend¬ 
ent  sample  testing.  Also,  the  independent 
sample  includes  cases  from  the  N.  Pacific. 
There  may  be  some  situations  unique  to 
that  region  that  are  not  handled  by  the 
rule  base  which  was  developed  using  only 
H.  Atlanl  data. 

Tl  .  t.SOP  forecast  performance  is  due 
to  foi  .  ist  skill,  as  evidenced  by  the 
Monte  •  .  lo  simulations.  The  independent 
sample  ii.-sults  demonstrate  considerable 
forecast  skill  for  very  difficult  fore¬ 
casting  situations.  AESOP  should  prove  to 
be  a  valuable  forecast  guidance  and  train¬ 
ing  tool  for  the  shipboard  meteorologist. 
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